Anatomical structure visualization systems and methods

ABSTRACT

In certain examples, an anatomical structure visualization system determines a distance between a point on an anatomical surface visible to an endoscope and a point on an embedded anatomical object that is visually occluded from the viewpoint of the endoscope by the anatomical surface. The anatomical structure visualization system may determine, based on the determined distance, a display parameter for a pixel of an image representative of a view of the anatomical surface from the viewpoint of the endoscope and assign the determined display parameter to the pixel of the image. The anatomical structure visualization system may similarly determine and assign display parameters to other pixels of the image and provide the image for display. The displayed image may provide a visualization of an anatomical structure at a surgical area, including a visualization of how deep the embedded anatomical object is positioned from the anatomical surface.

RELATED APPLICATIONS

The present application claims priority to U.S. Provisional PatentApplication No. 62/752,457, filed on Oct. 30, 2018, and entitled“ANATOMICAL STRUCTURE VISUALIZATION SYSTEMS AND METHODS,” the contentsof which are hereby incorporated by reference in their entirety.

BACKGROUND INFORMATION

During a computer-assisted surgical procedure, such as a minimallyinvasive surgical procedure that uses a computer-assisted surgicalsystem, an endoscope may be used to capture endoscopic imagery of asurgical area. The computer-assisted surgical system may display thecaptured endoscopic imagery to medical personnel (e.g., to a surgeonand/or other members of a surgical team) to provide a visualization ofthe surgical area. The visualized surgical area assists the medicalpersonnel in performing the surgical procedure. However, there remainsroom to improve visualizations of surgical areas and technologies usedto provide visualizations of surgical areas during a surgical procedure.

SUMMARY

An exemplary system includes a processor and a memory communicativelycoupled to the processor and storing instructions executable by theprocessor to determine a distance, along a line extending from aviewpoint of an endoscope in a three-dimensional space, between a pointon an anatomical surface visible to the endoscope and a point on anembedded anatomical object visually occluded from the viewpoint of theendoscope by the anatomical surface, determine, based on the determineddistance, a display parameter for a pixel of an image representative ofa view of the anatomical surface from the viewpoint of the endoscope,the pixel corresponding to the point on the anatomical surface, andassign the determined display parameter to the pixel of the image.

An exemplary computer-assisted surgical system includes at least onephysical computing device communicatively coupled to a stereoscopicendoscope and a display device, the at least one physical computingdevice configured to determine a distance between a point on ananatomical surface visible to the endoscope and a point on an embeddedanatomical object visually occluded from the viewpoint of the endoscopeby the anatomical surface, determine, based on the determined distance,a display parameter for a pixel of an image representative of a view ofthe anatomical surface from the viewpoint of the endoscope, the pixelcorresponding to the point on the anatomical surface, and provide theimage for display by the display device, the image including the pixeldisplayed in accordance with the display parameter.

An exemplary method includes determining, by an anatomical structurevisualization system, a distance, along a line extending from aviewpoint of an endoscope in a three-dimensional space, between a pointon an anatomical surface visible to the endoscope and a point on amodeled anatomical object visually occluded from the viewpoint of theendoscope by the anatomical surface, determining, by the anatomicalstructure visualization system and based on the determined distance, adisplay parameter for a pixel of an image representative of a view ofthe anatomical surface from the viewpoint of the endoscope, the pixelcorresponding to the point on the anatomical surface, and assigning, bythe anatomical structure visualization system, the determined displayparameter to the pixel of the image.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings illustrate various embodiments and are a partof the specification. The illustrated embodiments are merely examplesand do not limit the scope of the disclosure. Throughout the drawings,identical or similar reference numbers designate identical or similarelements.

FIG. 1 illustrates an exemplary computer-assisted surgical systemaccording to principles described herein.

FIG. 2 illustrates an exemplary stereoscopic endoscope located at asurgical area associated with a patient according to principlesdescribed herein.

FIG. 3 illustrates an exemplary anatomical structure visualizationsystem according to principles described herein.

FIG. 4 illustrates an exemplary configuration that includes ananatomical structure of which the anatomical structure visualizationsystem of FIG. 1 may provide a visualization according to principlesdescribed herein.

FIGS. 5-6 illustrate exemplary functions specifying how a displayparameter changes for various distances between surface anatomy andembedded anatomy according to principles described herein.

FIG. 7 illustrates an exemplary anatomical structure visualizationmethod according to principles described herein.

FIG. 8 illustrates an exemplary computing device according to principlesdescribed herein.

DETAILED DESCRIPTION

Anatomical structure visualization systems and methods are describedherein. Exemplary anatomical structure visualization systems and methodsdescribed herein may be implemented by a computer-assisted surgicalsystem and may provide a visualization of an anatomical structure at asurgical area during a surgical procedure. As described herein, thevisualization may represent and visually indicate depth of theanatomical structure, such as the depth of an embedded anatomical objectfrom an anatomical surface of the anatomical structure, in a manner thatis intuitive and helpful to a surgical team member such as a surgeon.

In certain examples, an anatomical structure visualization system maydetermine a distance between a point on an anatomical surface visible toan endoscope and a point on an embedded anatomical object that isvisually occluded from the viewpoint of the endoscope by the anatomicalsurface. The anatomical structure visualization system may determine,based on the determined distance, a display parameter for a pixel of animage representative of a view of the anatomical surface from theviewpoint of the endoscope and assign the determined display parameterto the pixel of the image. The anatomical structure visualization systemmay similarly determine and assign display parameters to other pixels ofthe image and provide the image for display. The displayed image mayprovide a visualization of an anatomical structure at a surgical area,including a visualization of how deep the embedded anatomical object ispositioned from the anatomical surface.

To this end, in certain examples, the display parameters of the pixelsof the image may be determined in a manner that increases, within animage, visual emphasis of an embedded anatomical object that isrelatively more proximate to the anatomical surface and decreases,within an image, visual emphasis of an embedded anatomical object thatis relatively less proximate to the anatomical surface. Thus, a degreeof visual emphasis of the embedded anatomical object in the image mayvisually represent depth of the embedded anatomical object from theanatomical surface of the anatomical structure. Examples of theanatomical structure visualization system determining display parametersof pixels of an image are described herein.

In certain examples, the anatomical structure visualization systemprovides user controls for use by a surgical team member to adjustsettings of the anatomical structure visualization system. As anexample, user controls may facilitate adjustment of settings that modifyhow depth of an embedded anatomical object is visually indicated in animage. For example, user controls may be used by a surgical team memberto adjust the display parameters of an image and/or how the displayparameters are determined by the anatomical structure visualizationsystem. In certain examples, for instance, user controls may allow asurgical team member to adjust a maximum depth within which an embeddedanatomical object is visually represented in an image and beyond whichan embedded anatomical object is not visually represented in an image, aminimum value of a display parameter (e.g., a minimum visibility) thatsets a minimum visual emphasis that is to be displayed for an embeddedanatomical object represented in an image, and/or a prominencemultiplier that adjusts a degree of visual emphasis that is used tovisually represent an embedded anatomical object in an image. In certainexamples, one or more of the user controls described herein mayfacilitate real-time adjustment of settings to allow settings to beadjusted by a surgical team member on-the-fly during a surgicalprocedure.

In certain examples, the anatomical structure visualization systemprovides user controls for use by a surgical team member to togglebetween display modes provided by the anatomical structure visualizationsystem. As an example, user controls may facilitate toggling between aplurality of display modes, which may include a mode for visualizationof only surface anatomy visible to an endoscope, a mode for concurrentvisualization of surface anatomy visible to the endoscope and embeddedanatomy that may be hidden from the view of the endoscope, and a modefor visualization of only embedded anatomy. In certain examples, one ormore of the user controls described herein may facilitate real-timetoggling of display modes to allow display modes to be toggled by asurgical team member on-the-fly during a surgical procedure.

Anatomical structure visualization systems and methods described hereinmay operate as part of or in conjunction with a computer-assistedsurgical system. As such, in order to promote an understanding ofanatomical structure visualization systems and methods described herein,an exemplary computer-assisted surgical system will now be described.The described exemplary computer-assisted surgical system isillustrative and not limiting. Anatomical structure visualizationsystems and methods described herein may operate as part of or inconjunction with the computer-assisted surgical system described hereinand/or with other suitable computer-assisted surgical systems.

FIG. 1 illustrates an exemplary computer-assisted surgical system 100(“surgical system 100”). As shown, surgical system 100 may include amanipulating system 102, a user control system 104, and an auxiliarysystem 106 communicatively coupled one to another. Surgical system 100may be utilized by a surgical team to perform a computer-assistedsurgical procedure on a patient 108. As shown, the surgical team mayinclude a surgeon 110-1, an assistant 110-2, a nurse 110-3, and ananesthesiologist 110-4, all of whom may be collectively referred to as“surgical team members 110.” Additional or alternative surgical teammembers may be present during a surgical session as may serve aparticular implementation.

While FIG. 1 illustrates an ongoing minimally invasive surgicalprocedure, it will be understood that surgical system 100 may similarlybe used to perform open surgical procedures or other types of surgicalprocedures that may similarly benefit from the accuracy and convenienceof surgical system 100. Additionally, it will be understood that thesurgical session throughout which surgical system 100 may be employedmay not only include an operative phase of a surgical procedure, as isillustrated in FIG. 1, but may also include preoperative, postoperative,and/or other suitable phases of the surgical procedure. A surgicalprocedure may include any procedure in which manual and/or instrumentaltechniques are used on a patient to investigate or treat a physicalcondition of the patient.

As shown in FIG. 1, manipulating system 102 may include a plurality ofmanipulator arms 112 (e.g., manipulator arms 112-1 through 112-4) towhich a plurality of surgical instruments may be coupled. Each surgicalinstrument may be implemented by any suitable surgical tool (e.g., atool having tissue-interaction functions), medical tool, monitoringinstrument (e.g., an endoscope), sensing instrument (e.g., aforce-sensing surgical instrument), diagnostic instrument, or the likethat may be used for a computer-assisted surgical procedure on patient108 (e.g., by being at least partially inserted into patient 108 andmanipulated to perform a computer-assisted surgical procedure on patient108). While manipulating system 102 is depicted and described herein asincluding four manipulator arms 112, it will be recognized thatmanipulating system 102 may include only a single manipulator arm 112 orany other number of manipulator arms as may serve a particularimplementation.

Manipulator arms 112 and/or surgical instruments attached to manipulatorarms 112 may include one or more displacement transducers, orientationalsensors, and/or positional sensors used to generate raw (i.e.,uncorrected) kinematics information. One or more components of surgicalsystem 100 may be configured to use the kinematics information to track(e.g., determine positions of) and/or control the surgical instruments.

Surgical instruments attached to manipulator arms 112 may each bepositioned at a surgical area associated with a patient. As used herein,a “surgical area” associated with a patient may, in certain examples, beentirely disposed within the patient and may include an area within thepatient near where a surgical procedure is planned to be performed, isbeing performed, or has been performed. For example, for a minimallyinvasive surgical procedure being performed on tissue internal to apatient, the surgical area may include the tissue, anatomy underlyingthe tissue, as well as space around the tissue where, for example,surgical instruments being used to perform the surgical procedure arelocated. In other examples, a surgical area may be at least partiallydisposed external to the patient. For instance, surgical system 100 maybe used to perform an open surgical procedure such that part of thesurgical area (e.g., tissue being operated on) is internal to thepatient while another part of the surgical area (e.g., a space aroundthe tissue where one or more surgical instruments may be disposed) isexternal to the patient. A surgical instrument may be referred to asbeing located at or within a surgical area when at least a portion ofthe surgical instrument (e.g., a distal end of the surgical instrument)is located within the surgical area.

User control system 104 may be configured to facilitate control bysurgeon 110-1 of manipulator arms 112 and surgical instruments attachedto manipulator arms 112. For example, surgeon 110-1 may interact withuser control system 104 to remotely move or manipulate manipulator arms112 and the surgical instruments. To this end, user control system 104may provide surgeon 110-1 with imagery (e.g., high-definition 3Dimagery) of a surgical area associated with patient 108 as captured byan endoscope. In certain examples, user control system 104 may include astereo viewer having two displays where stereoscopic images of asurgical area associated with patient 108 and generated by astereoscopic endoscope may be viewed by surgeon 110-1. Surgeon 110-1 mayutilize the imagery to perform one or more procedures with one or moresurgical instruments attached to manipulator arms 112.

To facilitate control of surgical instruments, user control system 104may include a set of master controls. These master controls may bemanipulated by surgeon 110-1 to control movement of surgical instruments(e.g., by utilizing robotic and/or teleoperation technology). The mastercontrols may be configured to detect a wide variety of hand, wrist, andfinger movements by surgeon 110-1. In this manner, surgeon 110-1 mayintuitively perform a procedure using one or more surgical instruments.

Auxiliary system 106 may include one or more computing devicesconfigured to perform primary processing operations of surgical system100. In such configurations, the one or more computing devices includedin auxiliary system 106 may control and/or coordinate operationsperformed by various other components (e.g., manipulating system 102 anduser control system 104) of surgical system 100. For example, acomputing device included in user control system 104 may transmitinstructions to manipulating system 102 by way of the one or morecomputing devices included in auxiliary system 106. As another example,auxiliary system 106 may receive, from manipulating system 102, andprocess image data representative of imagery captured by an endoscopeattached to one of manipulator arms 112.

In some examples, auxiliary system 106 may be configured to presentvisual content to surgical team members 110 who may not have access tothe images provided to surgeon 110-1 at user control system 104. To thisend, auxiliary system 106 may include a display monitor 114 configuredto display one or more user interfaces, such as images (e.g., 2D images)of the surgical area, information associated with patient 108 and/or thesurgical procedure, and/or any other visual content as may serve aparticular implementation. For example, display monitor 114 may displayimages of the surgical area together with additional content (e.g.,graphical content, contextual information, etc.) concurrently displayedwith the images. In some embodiments, display monitor 114 is implementedby a touchscreen display with which surgical team members 110 mayinteract (e.g., by way of touch gestures) to provide user input tosurgical system 100.

Manipulating system 102, user control system 104, and auxiliary system106 may be communicatively coupled one to another in any suitablemanner. For example, as shown in FIG. 1, manipulating system 102, usercontrol system 104, and auxiliary system 106 may be communicativelycoupled by way of control lines 116, which may represent any wired orwireless communication link as may serve a particular implementation. Tothis end, manipulating system 102, user control system 104, andauxiliary system 106 may each include one or more wired or wirelesscommunication interfaces, such as one or more local area networkinterfaces, Wi-Fi network interfaces, cellular interfaces, etc.

FIG. 2 illustrates an exemplary stereoscopic endoscope 200. Endoscope200 may be manually controlled (e.g., by a surgeon performing a surgicalprocedure on a patient). Alternatively, endoscope 200 may be coupled toa manipulator arm (e.g., one of manipulator arms 112) of acomputer-assisted surgical system (e.g., surgical system 100), andcontrolled using robotic and/or teleoperation technology. Endoscope 200is representative of many different types and/or implementations ofendoscopes that may be used with systems and methods described herein.

As shown, endoscope 200 includes a shaft 202 and a camera head 204coupled to a proximal end of shaft 202. Camera head 204 is configured tobe located external to the patient. Shaft 202 has a distal end that isconfigured to be positioned at (e.g., inserted into) a surgical area ofa patient. In various implementations, shaft 202 is rigid (as shown inFIG. 2). Alternatively, shaft 202 may be jointed and/or flexible.

As shown, camera head 204 houses a right-side camera control unit 206-R,a left-side camera control unit 206-L, and an illuminator 208. In somealternative examples, camera control units 206 and illuminator 208 arenot included in camera head 204 and are instead located in an endoscopecontroller communicatively coupled to endoscope 200. The endoscopecontroller may be implemented by auxiliary system 106, for example.

Shaft 202 houses a right-side image sensor 210-R optically coupled to aright-side optic 212-R, a left-side image sensor 210-L optically coupledto a left-side optic 212-L, and an illumination channel 214. Theright-side components (i.e., camera control unit 206-R, image sensor210-R, and optic 212-R) implement a camera that captures images 216-R ofthe surgical area from a right-side perspective. Likewise, the left-sidecomponents (i.e., camera control unit 206-L, image sensor 210-L, andoptic 212-L) implement a camera that captures images 216-L of thesurgical area from a left-side perspective.

To capture images 216, illuminator 208 generates light, which is carriedby one or more optical fibers in illumination channel 214 and outputinto the surgical area at a distal end of shaft 202. Optics 212, whichmay each be implemented by a lens or other suitable component, capturethe light after the light reflects from patient anatomy and/or otherobjects within the surgical area.

The light captured by optics 212 is sensed by image sensors 210. Imagesensors 210 may be implemented as any suitable image sensors such ascharge coupled device (“CCD”) image sensors, complementary metal-oxidesemiconductor (“CMOS”) image sensors, or the like. Image sensors 210-Rand 210-L convert the sensed light into signals (e.g., video data)representative of images, and transmit the signals to camera controlunits 206 by way of conduits 218-R and 218-L, respectively. Conduits 218may be any suitable communication link configured to handle high-speedtransmission of data.

Camera control units 206 process the signals received from image sensors210 and generate, based on the signals, data representative of images216. Camera control units 206 then transmit the data to an externaldevice (e.g., a computing device that processes the images and/ordisplays the images and/or video formed by the images on a displayscreen). As shown, camera control units 206 are synchronously coupled toone another by way of a communicative link 220 so that images 216 aresynchronized.

Additional or alternative components may be included in endoscope 200.For example, one or more other optics not explicitly shown in FIG. 2 maybe included in shaft 202 for focusing, diffusing, or otherwise treatinglight generated and/or sensed by endoscope 200. In some alternativeexamples, image sensors 210 can be positioned closer to the proximal endof shaft 202 or inside camera head 204, a configuration commonlyreferred to as a rod lens endoscope.

Endoscope 200 may provide data representing visible light data of asurgical area. For example, endoscope 200 may capture visible lightimages of the surgical area that represent visible light sensed byendoscope 200. Visible light images may include images that use anysuitable color and/or grayscale palette to represent a visible lightbased view of the surgical area.

Endoscope 200 may also provide data representing depth data of asurgical area or data that may be processed to derive depth data of thesurgical area. For example, endoscope 200 may capture images of thesurgical area that represent depth sensed by endoscope 200.Alternatively, endoscope 200 may capture images of the surgical areathat may be processed to derive depth data of the surgical area. Forexample, images 216-R and 216-L may be stereoscopic images of thesurgical area, which images may be processed to determine depthinformation for the surgical area. The depth information may berepresented as depth images (e.g., depth map images obtained using aZ-buffer that indicates distance from endoscope 200 to each pixel pointon an image of a surgical area), which may be configured to visuallyindicate depths of objects in the surgical area in any suitable way,such as by using different greyscale values to represent different depthvalues.

Images captured by an endoscope (e.g., by endoscope 200) and/or derivedfrom images captured by endoscope 200 (e.g., visible light images anddepth images) may be referred to as “endoscopic imagery.” Exemplaryanatomical visualization systems and methods described herein may beconfigured to utilize endoscopic imagery to provide visualizations ofanatomical structures, such as described herein.

Endoscope 200 shown in FIG. 2 is illustrative of one imaging device thatmay be used to obtain endoscopic imagery. Any other suitable imagingdevice or combination of devices from which visible light data and depthdata of a surgical area may be obtained or derived during a surgicalprocedure may be used in other examples.

FIG. 3 shows an exemplary anatomical structure visualization system 300(“visualization system 300” or “system 300”) configured to provide avisualization of an anatomical structure at a surgical area for use byone or more surgical team members during a surgical procedure. As shown,system 300 may include, without limitation, a distance determinationfacility 302, a display facility 304, a user control facility 306, and astorage facility 308 selectively and communicatively coupled to oneanother. It will be recognized that although facilities 302 through 308are shown to be separate facilities in FIG. 3, facilities 302 through308 may be combined into fewer facilities, such as into a singlefacility, or divided into more facilities as may serve a particularimplementation. Each of facilities 302 through 308 may be implemented byany suitable combination of computing hardware and/or software. Inalternative embodiments, one or more of facilities 302 through 308 maybe omitted from system 300 and/or one or more additional facilities maybe included in system 300.

System 300 may be associated with a computer-assisted surgical systemsuch as surgical system 100 in any suitable manner. For example, system300 may be implemented by or included within a computer-assistedsurgical system. To illustrate, system 300 may be implemented by one ormore computing devices included within manipulating system 102, usercontrol system 104, and/or auxiliary system 106 of surgical system 100.In some examples, system 300 may be at least partially implemented byone or more computing devices communicatively coupled to, but notincluded in, a computer-assisted surgical system (e.g., one or moreservers communicatively coupled to surgical system 100 by way of anetwork).

Distance determination facility 302 may be configured to perform one ormore operations to determine distances between surface anatomy that isvisible to an endoscope and embedded anatomy that may be hidden from theview of the endoscope by the surface anatomy. As described herein, sucha distance may be a linear distance, within a 3D space, between a pointon the surface anatomy and a point on the embedded anatomy. As will alsobe described herein, the point on the surface anatomy and the point onthe embedded anatomy may be associated with a pixel of an image, andsystem 100 may utilize the determined distance between the points todetermine a display parameter for the pixel of the image.

In certain examples, to facilitate determining distances between surfaceanatomy and embedded anatomy, distance determination facility 302 mayaccess endoscopic imagery of a surgical area of a patient, which imagerymay include visible light images, depth images, and metadata for theimages. The metadata may include any information associated with theimages, such as a position of an endoscope from which endoscopic imagesare captured, camera parameters (e.g., intrinsic and/or extrinsic cameraparameters), a reference frame of the endoscope, etc. Distancedetermination facility 302 may access the endoscopic imagery in anysuitable way from any suitable source, such as directly or indirectlyfrom an endoscope that captures images of a surgical area in real timeduring a surgical procedure.

Distance determination facility 302 may also access model datarepresentative of modeled anatomy of the patient. The model data mayrepresent a model of anatomy associated with the surgical area of thepatient, such as a model of embedded anatomy that may be hidden from theview of the endoscope by surface anatomy. The model of the anatomy maybe generated at any suitable time, including in advance of or as part ofa surgical procedure and/or in advance of or concurrently with thecapture of endoscopic imagery. The model of the anatomy may include anysuitable three-dimensional model of the anatomy represented in anysuitable data format (e.g., using a Digital Imaging and Communicationsin Medicine (DICOM) standard specifying a file format definition and/ornetwork communication protocol for medical imaging data). The model mayinclude a model, or a set of segmented models, generated from anysuitable 3D imaging procedure, such as a computed tomography (CT) scan,a segmented DICOM scan, a magnetic resonance imaging (MRI) scan,fluorescence imaging, infrared or near-infrared imaging, imaging showingdye or other markers, or the like.

Distance determination facility 302 may register the endoscopic imageryand the model of anatomy to a common 3D space and/or reference frame inany suitable way, such as by aligning anatomical anchor points in theendoscopic imagery and/or modeled anatomy. In certain examples, distancedetermination facility 302 may perform one or more operations toregister the model of the anatomy with a reference frame of theendoscope that captured the endoscopic imagery.

By registering the endoscopic imagery and the model of anatomy to acommon 3D space and/or reference frame, distance determination facility302 may generate a converged model in which both the endoscopic imageryand the model of anatomy are represented in a common 3D space. In otherexamples, registration of the endoscopic imagery and the model ofanatomy to a common 3D space and/or reference frame may be performedoutside of system 300. In such examples, distance determination facility302 may simply access data representative of the converged model from asuitable source.

Distance determination facility 302 may use the converged model todetermine distances between surface anatomy visible to the endoscope andmodeled anatomy, such as embedded anatomy that is occluded from the viewof the endoscope by the surface anatomy. Distance determination facility302 may determine such distances in any suitable way. An exemplary wayof determining such distances will now be described with reference toFIG. 4.

FIG. 4 illustrates an exemplary configuration 400 that includes ananatomical structure 402 of which system 100 may provide avisualization. As shown, anatomical structure 402 includes surfaceanatomy 404 and embedded anatomy 406. Surface anatomy 404 includes ananatomical surface that may be visible to an endoscope 408 positioned asshown in FIG. 4 (e.g., positioned at a surgical area during a surgicalprocedure). The anatomical surface may include at least a portion ofsurface anatomy 404 that is within a field of view 410 of endoscope 408.Embedded anatomy 406 may include an anatomical object that is embeddedwithin surface anatomy 404 or otherwise occluded from the view ofendoscope 408 by surface anatomy 404.

Anatomical structure 402 may be represented by a converged model asdescribed above. For example, endoscope 408 may capture endoscopicimagery of the anatomical surface that is visible to endoscope. Usingdepth data included in or otherwise associated with the endoscopicimagery, a 3D model of the anatomical surface may be generated, such asa depth map of the anatomical surface. Distance determination facility302 may register the 3D model of the anatomical surface and a 3D modelof the embedded anatomical object to a common 3D space or referenceframe to form a converged model that represents the anatomical structure402 in the common 3D space or reference frame.

Distance determination facility 302 may use the converged modelrepresenting anatomical structure 402 to determine distances betweensurface anatomy 404 visible to endoscope 408 and embedded anatomy 406.In certain examples, Distance determination facility 302 may determinesuch a distance along a line extending from a viewpoint of endoscope 408in the common 3D space. For example, dashed line 412-1 represents a lineextending from the viewpoint of endoscope 408 and intersecting a pointP1 on surface anatomy 404 and a point P2 on embedded anatomy 406.Distance determination facility 302 may determine a distance D1 betweenpoint P1 on surface anatomy 404 and point P2 on embedded anatomy 406.Distance determination facility 302 may similarly determine distancesalong other lines extending from the viewpoint of endoscope 408 in thecommon 3D space. For example, dashed line 412-2 represents a lineextending from the viewpoint of endoscope 408 and intersecting a pointP3 on surface anatomy 404 and a point P4 on embedded anatomy 406.Distance determination facility 302 may determine a distance D2 betweenpoint P3 on surface anatomy 404 and point P4 on embedded anatomy 406.

In certain examples, lines 412-1 and 412-2 may represent imageprojection rays, such as perspective image projection rays projectedfrom a viewpoint of endoscope 408 and through points (e.g., pixels) onan image plane 414. In such examples, distances D1 and D2 may representdepths, along perspective image projection rays, from surface anatomy404 to embedded anatomy 406.

FIG. 4 illustrates one exemplary way that distance determinationfacility 302 may determine distances between surface anatomy 404 visibleto an endoscope and embedded anatomy 406 occluded from the view of theendoscope by surface anatomy 404. In other examples, distancedetermination facility 302 may be configured to determine distancesbetween surface anatomy 404 visible to an endoscope and embedded anatomy406 occluded from the view of the endoscope by surface anatomy 404 inany other suitable way, such as by using orthographic projection rays aslines along which to identify corresponding points on surface anatomy404 and embedded anatomy 406 and to determine distances between thecorresponding points.

Returning to FIG. 3, display facility 304 may be configured to provide avisualization of an anatomical structure based on distances, determinedby distance determination facility 302, between surface anatomy visibleto an endoscope and embedded anatomy occluded from the view of theendoscope by the surface anatomy. To this end, for example, displayfacility 304 may be configured to determine, based on determineddistances between surface anatomy and embedded anatomy included in ananatomical structure, display parameters for pixels of an imagerepresentative of a view of the anatomical structure, such as an imagerepresentative of a view of the surface anatomy from the viewpoint of anendoscope.

FIG. 4 shows pixels PX1 and PX2 included in image plane 414. Pixel PX1,point P1, and point P2 are intersected by line 412-1, and pixel PX2,point P3, and point P4 are intersected by line 412-2. Based on theselinear relationships, pixel PX1 may be said to correspond to point P1 onsurface anatomy 404 and point P2 on embedded anatomy 406, and pixel PX2may be said to correspond to point P3 on surface anatomy 404 and pointP4 on embedded anatomy 406. Display facility 304 may determine a displayparameter for pixel PX1 based on distance D1 between points P1 and P2and on display parameters associated with points P1 and/or P2. Displayfacility 304 may similarly determine a display parameter for pixel PX2based on distance D2 between points P3 and P4 and on display parametersassociated with points P3 and/or P4.

For a pixel on image plane 414 that corresponds to a point on surfaceanatomy 404 and a point on embedded anatomy 406, display facility 304may determine a display parameter for the pixel in a manner configuredto selectively visualize the point on embedded anatomy 406 as beingembedded within surface anatomy 404. The display parameter may includeany parameter configured to differentiate the visual appearance of thepixel from one or more other pixels of an image so as to visuallyindicate an existence and/or one or more properties of the point onembedded anatomy 406. The display parameter may include a color, anopacity, a transparency, a saturation, a brightness, and/or any otherdisplay parameter for the pixel. The display parameter may also includeany combination or sub-combination of such parameters.

In certain examples, display facility 304 may be configured to determinea display parameter for a pixel of the image based on a determineddistance and using a defined function that specifies how the displayparameter changes for different values of the determined distance. Thedefined function may be any suitable linear or non-linear function andmay be defined as may suit particular implementation.

FIG. 5 illustrates a graph 500 of a linear function 502 that specifiesvalues of a display parameter to be determined for different distances.FIG. 6 illustrates a graph 600 of a non-linear function 602 thatspecifies values of a display parameter to be determined for differentdistances. Functions 502 and 602 are illustrative of certain examples.Any other suitable function that specifies how a display parameterchanges for various determined distances may be used in other examples.For example, any suitable falloff function may be used that definesrelationships between a display parameter and distances betweenanatomical 3D models.

In certain examples, display facility 304 may be configured to determinea display parameter such as a color for a pixel of an image based on adefined color blending function. For example, using the color blendingfunction and based on a determined distance between a point on surfaceanatomy 404 and a point on embedded anatomy 406, display facility 304may blend a color associated with the point on embedded anatomy 406 witha color associated with the point on surface anatomy 404 to determinethe color for the pixel of the image. The color blending function may bedefined to give the color associated with the point on embedded anatomy406 more weight when the determined distance is relatively shorter andless weight when the determined distance is relatively longer. Thus, fora relatively shorter distance, the color associated with the point onembedded anatomy 406 may be emphasized more in the determined color forthe pixel of the image than for a relatively longer distance. To thisend, the color blending function may specify how the weight given to thecolor associated with the point on embedded anatomy 406 changes fordifferent values of the determined distance.

In certain examples, display facility 304 may be configured to determinea display parameter for a pixel of an image based on a parameter of acorresponding point on embedded anatomy 406. To this end, for example,display facility 304 may be configured to determine a blend parameterfor the point on embedded anatomy 406 based on the determined distancebetween the point on embedded anatomy 406 and a corresponding point onsurface anatomy 404. For example, using a defined blend function andbased on the determined distance, display facility 304 may determine ablend parameter for the point on embedded anatomy 406. The blendfunction may specify how the blend parameter changes for differentvalues of the determined distance.

Once the blend parameter for the point on embedded anatomy 406 isdetermined, display facility 304 may determine the display parameter forthe pixel of the image based on the blend parameter. In certainexamples, display facility 304 may be configured to give more weight toone or more display parameters associated with the point on embeddedanatomy 406 when the determined blend parameter for the point isrelatively higher (e.g., based on a relatively shorter determineddistance) and less weight to the one or more display parametersassociated with the point when the determined blend parameter for thepoint if relatively lower (e.g., based on a relatively larger determineddistance). Thus, for a relatively higher blend parameter, a displayparameter associated with the point on embedded anatomy 406 may beemphasized more in the determined display parameter for the pixel of theimage than for a relatively lower blend parameter.

In certain examples, display facility 304 may be configured to determinea display parameter for a pixel of an image based on a defined maximumdistance. For example, for a point on embedded anatomy 406, displayfacility 304 may be configured to determine the display parameter forthe corresponding pixel of the image based on a maximum distance beyondwhich the point on embedded anatomy 406 is not visually represented bythe pixel of the image and within which the point on embedded anatomy406 is visually represented by the pixel of the image. To this end, incertain examples, display facility 304 may be configured not to blend adisplay parameter of the point on embedded anatomy 406 with a displayparameter of a corresponding point on surface anatomy 404 when thedetermined distance between the points is greater than the definedmaximum distance, such that the point on embedded anatomy 406 will notbe represented in the pixel of the image when the point on embeddedanatomy 406 is too deep, i.e., beyond the maximum distance, from thecorresponding point on surface anatomy 404. Thus, the maximum distancemay provide a limited depth range within which embedded anatomy 406 willbe visually represented in an image and beyond which embedded anatomy406 will not be visually represented in the image.

In certain examples, display facility 304 may be configured to determinea display parameter for a pixel of an image based on a defined minimumvisualization threshold for embedded anatomy. For example, for a pointon embedded anatomy 406, display facility 304 may be configured todetermine the display parameter for the corresponding pixel of the imagebased on a minimum visualization threshold at which the point onembedded anatomy 406 may be visually represented by the pixel of theimage. To this end, in certain examples, display facility 304 may beconfigured to determine the display parameter to provide a visualemphasis that provides at least he minimum visualization threshold. Incertain examples, display facility 304 may be configured to satisfy theminimum visualization threshold using a minimum visibility parametersuch as a minimum opacity parameter allowed for the point on embeddedanatomy 406. In such examples, display facility 304 may set the opacityparameter for the point on embedded anatomy 406 to at least satisfy theminimum allowed opacity parameter.

In certain examples, any combination or sub-combination of colorblending functions, color parameters, blend coefficient functions (e.g.,opacity functions), blend parameters (e.g., opacity parameters), maximumdistance thresholds, minimum visualization thresholds, and/or otherdisplay parameters, display parameter functions, etc. may be used bydisplay facility 304 to determine a display parameter for a pixel of animage in a manner that selectively and visually represents embeddedanatomy together with surface anatomy.

In certain examples, for instance, display facility 304 may beconfigured to determine a color for a pixel of an image based on thefollowing algorithm:

PixelColor=PixelAColor.Blend(AColor, EColor, BlendCoeff).

In the algorithm, “PixelColor” is the determined color for the pixel,“PixelAColor.Blend” is a color blending function, “AColor” is a color ofa pixel point on surface anatomy captured by an endoscope, “EColor” is acolor of a corresponding pixel point on a model of embedded anatomy, and“BlendCoeff” is a color blending coefficient that may be determined bydisplay facility 304. Based on the algorithm, display facility 304 mayblend colors (e.g., RGB values) of the color of the pixel point on thesurface anatomy with the color of the pixel point on the embeddedanatomy based on blend coefficient “BlendCoeff.”

In certain examples, display facility 304 may determine “BlendCoeff”based on the following algorithm:

BlendCoeff=Max(MinCE, FalloffFunction(D, MaxD)).

In this blend coefficient algorithm, “BlendCoeff” is the determinedblend coefficient, “MinCE” is a minimal level of visibility for theembedded anatomy, “D” is a determined distance between the point on thesurface anatomy and the corresponding point on the embedded anatomy,“MaxD” is a maximum depth to normalize distance to (e.g., representinghow deep into tissue will be visualized), “FalloffFunction” is afunction the defines how the color of embedded anatomy decreases as thedetermined distance “D” gets closer to the maximum depth “MaxD” (e.g.,how the determined distance “D” is translated to a value between 0-1),and “Max” will not allow BlendCoeff to become smaller than MinCE.

The above-described algorithms illustrate one example of how displayfacility 304 may determine a color for a pixel of an image in a mannerthat visualizes depth of embedded anatomy together with surface anatomy.Other suitable algorithms for determining a display parameter for apixel of an image to visualize depth of embedded anatomy may be used inother examples. Such algorithms may blend display parameter values fromtwo or more anatomical 3D model layers to determine a blended displayparameter (e.g. a blended color), the blended display parameterdetermined based on a depth or depths between the 3D model layers.

Once display facility 304 determines a display parameter (e.g., ablended color) for a pixel of an image, display facility 304 may assignthe display parameter to the pixel of the image. The assignment may bemade in any way suitable for rendering and/or display of the pixel ofthe image to be based on the assigned display parameter.

Display facility 304 may determine and assign display parameters for allpixels of an image. Display facility 304 may provide the image forrendering and/or display by a display device. When displayed, the imagemay provide a visualization of depths of embedded anatomy from surfaceanatomy, including a visualization of differences in depths betweendifferent points on embedded anatomy.

Returning to FIG. 4, distance D2 is shorter than distance D1.Accordingly, display facility 304 may determine and assign displayparameters to pixels PX2 and PX1 in a way that will visually emphasize,within an image, point P4 more than point P2 based on point P4 beingcloser than point P2 to surface anatomy 404 from the perspective thatthe image is rendered (e.g., from the perspective of endoscope 408). Forexample, a color of point P4 may be blended into the color of pixel PX2more than a color of point P2 is blended into the color of pixel PX1,which may visually emphasis point P4 more than point P2 in the image.

By basing the visual emphasis of embedded anatomy 406 on point-by-pointdistances, such as distances D1 and D2, between embedded anatomy 406 andsurface anatomy 404, display facility 304 may determine displayparameters for pixels of an image in a manner that may visualize thepoint-by-point depths of embedded anatomy 406 in the image. In certainexamples, such an image may provide a visualization of a pixel-by-pixeltopographic representation of embedded anatomy 406 in the image,together with a visualization of surface tissue captured by anendoscope. In this or another manner, the image may provide avisualization of an anatomical structure having anatomical componentsthat are visible to and other anatomical components that are hidden fromthe viewpoint of an endoscope located at a surgical site during asurgical procedure.

In certain implementations, display facility 304 may provide an imagefor display as part of a sequence of images such as video images of asurgical area. Display facility 304 may be configured to generate videoimages in real time such that the video images visually representreal-time depths of embedded anatomy 406 relative to surface anatomy 404visible to endoscope 408. Accordingly, as a surgical team membercontrols and moves endoscope 408 (e.g., a position and/or orientation ofendoscope 408) relative to a surgical area, system 100 may continuallydetermine distances between embedded anatomy 406 and surface anatomy 404and determine, based on the distances, display parameters for pixels ofvideo images such that each frame of the video provides a real-timevisualization of depths of embedded anatomy 406 relative to surfaceanatomy 404.

Returning to FIG. 3, user control facility 306 may be configured toprovide one or more controls for use by a user of system 300 to adjustone or more settings used by display facility 304 to determine a displayparameter based on a determined distance between embedded anatomy andsurface anatomy. The user controls may be provided in any way suitablefor use by a user of system 300, such as for use by a surgical teammember who uses computer-assisted surgical system 100, to provide userinput to adjust one or more settings used by display facility 304 todetermine a display parameter. In certain examples, the user controlsmay be configured to facilitate user input to adjust the displayparameter in real time during a surgical procedure in order to modifyhow depth of embedded anatomy is visually indicated in an image. In someimplementations, the user controls may be provided as controls (e.g., asmaster controls) of user control system 104 of computer-assistedsurgical system 100. For example, such user controls may be provided, onuser control system 104, as a slider user input mechanism thatfacilitates user input to adjust a display parameter setting to a valuewithin a range of available values for the display parameter. Such aslider input mechanism may facilitate a user-controlled gradualtransition (e.g. a fade-in/fade out of embedded anatomy) or crossfadebetween visualization of surface anatomy and embedded anatomy.

As an example, a user control may allow a surgical team member to adjusta maximum depth (e.g., MaxD in an algorithm described above) withinwhich embedded anatomy is visually represented in an image and beyondwhich an embedded anatomy is not visually represented in an image. Thismay allow a surgical team member to control how deep beyond visiblesurface tissue that embedded anatomy will be visualized. As anotherexample, a user control may allow a surgical team member to adjust aminimum visualization threshold (e.g., MinCE in an algorithm describedabove) for embedded anatomy. This may allow the surgical team member tocontrol a minimum allowable visual emphasis (e.g., a minimum opacity)that is to be displayed for embedded anatomy. As another example, a usercontrol may allow a surgical team member to adjust a prominencemultiplier that adjusts a degree of visual emphasis that is used tovisually represent embedded anatomy in an image. This may allow thesurgical team member to control how prominently embedded anatomy isvisually emphasized in an image.

By providing such user controls for use by a surgical team member, thesurgical team member may intuitively and conveniently adjust settingsfor visualization of embedded anatomy to fit a particular situation,configuration, and/or need during a surgical procedure. For example, ifa surgeon wants to focus attention on surface anatomy and embeddedanatomy immediately beneath surface anatomy, the surgeon may utilize auser control to adjust a maximum distance threshold (e.g., MaxD) to asmall value that will result in only embedded anatomy immediatelybeneath surface anatomy being visualized in an image.

In certain examples, user control facility 306 may be configured toprovide one or more controls for use by a user of system 300 to togglebetween different display modes provided by system 300 and/orcomputer-assisted surgical system 100. The user controls may be providedin any way suitable for use by a user of system 300, such as for use bya surgical team member who uses computer-assisted surgical system 100,to provide user input to toggle between display modes. In certainexamples, the user controls may be configured to facilitate user inputto toggle between display modes in real time during a surgicalprocedure. In some implementations, a toggle, slide, or other controlmay be provided as a control of (e.g., a master control of) user controlsystem 104 of computer-assisted surgical system 100.

In certain examples, a user control for toggling between display modesmay facilitate user input to toggle between a mode for visualization ofonly surface anatomy visible to an endoscope, a mode for concurrentvisualization of surface anatomy visible to the endoscope and embeddedanatomy, and a mode for visualization of only embedded anatomy. In themode for visualization of only visible surface anatomy, display facility304 may provide images that display representations of surface anatomythat is visible to an endoscope without incorporating any visualrepresentation of embedded anatomy that is occluded from the view of theendoscope. In the mode for concurrent visualization of surface anatomyvisible to the endoscope and embedded anatomy, display facility 304 mayprovide images that display visual representations of surface anatomythat is visible to an endoscope together with visual representations ofanatomy embedded within the surface anatomy or otherwise hidden from theview of the endoscope by the surface anatomy. In the mode forvisualization of only embedded anatomy, display facility 304 may provideimages that display representations of embedded anatomy withoutrepresentations of anatomy that is visible to an endoscope. In the modefor visualization of only embedded anatomy, system 300 and/or system 100may provide one or more user control lockout features that prevent auser from using certain user controls while only embedded anatomy isdisplayed, so as to help prevent an error from being made while surfaceanatomy that is visible to an endoscope is not being displayed.

Storage facility 308 may store any data received, generated, managed,maintained, used, and/or transmitted by facilities 302 through 306 in aparticular implementation. For example, storage facility 308 may storeprogram instructions (e.g., computer code) for performing operationsdescribed herein, model data representing a model of anatomy (e.g., a 3Dmodel of anatomy), endoscopic imagery data representing endoscopicimagery, data representing determined display parameters, image datarepresenting generated images, and/or any other data as may serve aparticular implementation.

FIG. 7 illustrates an exemplary anatomical structure visualizationmethod 700. While FIG. 7 illustrates exemplary operations according toone embodiment, other embodiments may omit, add to, reorder, and/ormodify any of the operations shown in FIG. 7. One or more of theoperations shown in FIG. 7 may be performed by an anatomical structurevisualization system such as system 300, any components includedtherein, and/or any implementation thereof.

In operation 702, an anatomical structure visualization system maydetermine distance between surface anatomy visible to an endoscope andembedded anatomy occluded from the endoscope by the surface anatomy.Operation 702 may be performed in any of the ways described herein,including by the anatomical structure visualization system using aconverged model that includes endoscopic imagery of a patient andmodeled anatomy of the patient registered to a common 3D space orreference frame to determine the distance, which may include a distance,along a line extending from a viewpoint of the endoscope in the 3Dspace, between a point on the surface anatomy and a point on the modeledanatomy.

In operation 704, the anatomical structure visualization system maydetermine and assign image display parameters based on the distancedetermined in operation 702. Operation 704 may be performed in any ofthe ways described herein, including by the anatomical structurevisualization system determining and assigning display parameters forpixels of an image representative of an anatomical structure, such as animage representative of a view of the surface anatomy from the viewpointof the endoscope.

In operation 706, the anatomical structure visualization system mayprovide the image, for display, based on the display parameters.Operation 704 may be performed in any of the ways described herein. Asdescribed herein, the image may provide a visualization of the embeddedanatomy together with the surface anatomy from the viewpoint of theendoscope, where the image visualizes how deep the embedded anatomy ispositioned from the surface anatomy.

Method 700 may be continually repeated to generate and provide, in realtime during a surgical procedure, images as frames of video of asurgical area during the surgical procedure. Accordingly, the video mayassist a surgical team member in performing the surgical procedure, suchas by visualizing, to the surgical team member in real time, how deepembedded anatomy not visible to an endoscope is located from surfaceanatomy visible to the endoscope.

While the video is being generated and presented in real time during thesurgical procedure, the surgical team member may utilize a user controlto adjust a setting used to determine how to represent the embeddedanatomy. In response to the adjusted setting, the anatomical structurevisualization system may modify, in real time how embedded anatomy isrepresented in the video, such as by changing how one or more displayparameters are determined for the video.

Certain operations are described herein as being able to be performed inreal time in some examples. Operations may be performed in real time ornear real time when they are performed immediately and without unduedelay such that, for example, data processing operations associated withan ongoing event or procedure (e.g., a surgical procedure) are performedwithout undue delay even if there is some amount of processing delay.

While certain examples described herein are directed to determining adisplay parameter for a pixel of an image based on a point on surfaceanatomy and a point on a modeled, embedded anatomical object, in certainexamples, system and methods described herein may determine a displayparameter for a pixel of an image, in any of the ways described herein,based on a point on surface anatomy and multiple points on one or moremodeled, embedded anatomical objects, depending on a number, shape,and/or configuration of modeled, embedded anatomical objects relative tothe surface anatomy and viewpoint of an endoscope.

In certain examples, the visualization of embedded anatomy, as describedherein, may assist a surgical team member in performing a surgicalprocedure in one or more ways. For example, a surgeon may intuitively,conveniently, and/or accurately navigate surgical tools at a surgicalarea and/or manipulate tissue at a surgical area. Visualizationsdescribed herein may present helpful information, such as locations ofembedded anatomical landmarks, in intuitive and natural ways that assistthe surgeon and without overloading the surgeon with too muchinformation.

Visualizations such as the exemplary visualizations described herein mayprovide one or more advantages and/or benefits compared to conventionaltechnologies for displaying multiple layers of anatomy, including any ofthe advantages and/or benefits described or made apparent herein.Certain conventional technologies that overlay modeled anatomy are notaware of and do not visualize depth of the modeled anatomy from surfaceanatomy. For example, using conventional technologies, a model of ablood vessel may be simply overlaid on imagery of surface tissue that isvisible to an endoscope without providing any information about how deepthe blood vessel is embedded behind the surface tissue.

In certain embodiments, one or more of the systems, components, and/orprocesses described herein may be implemented and/or performed by one ormore appropriately configured computing devices. To this end, one ormore of the systems and/or components described above may include or beimplemented by any computer hardware and/or computer-implementedinstructions (e.g., software) embodied on at least one non-transitorycomputer-readable medium configured to perform one or more of theprocesses described herein. In particular, system components may beimplemented on one physical computing device or may be implemented onmore than one physical computing device. Accordingly, system componentsmay include any number of computing devices, and may employ any of anumber of computer operating systems.

In certain embodiments, one or more of the processes described hereinmay be implemented at least in part as instructions embodied in anon-transitory computer-readable medium and executable by one or morecomputing devices. In general, a processor (e.g., a microprocessor)receives instructions, from a non-transitory computer-readable medium,(e.g., a memory, etc.), and executes those instructions, therebyperforming one or more processes, including one or more of the processesdescribed herein. Such instructions may be stored and/or transmittedusing any of a variety of known computer-readable media.

A computer-readable medium (also referred to as a processor-readablemedium) includes any non-transitory medium that participates inproviding data (e.g., instructions) that may be read by a computer(e.g., by a processor of a computer). Such a medium may take many forms,including, but not limited to, non-volatile media, and/or volatilemedia. Non-volatile media may include, for example, optical or magneticdisks and other persistent memory. Volatile media may include, forexample, dynamic random access memory (“DRAM”), which typicallyconstitutes a main memory. Common forms of computer-readable mediainclude, for example, a disk, hard disk, magnetic tape, any othermagnetic medium, a compact disc read-only memory (“CD-ROM”), a digitalvideo disc (“DVD”), any other optical medium, random access memory(“RAM”), programmable read-only memory (“PROM”), electrically erasableprogrammable read-only memory (“EPROM”), FLASH-EEPROM, any other memorychip or cartridge, or any other tangible medium from which a computercan read.

FIG. 8 illustrates an exemplary computing device 800 that may bespecifically configured to perform one or more of the processesdescribed herein. As shown in FIG. 8, computing device 800 may include acommunication interface 802, a processor 804, a storage device 806, andan input/output (“I/O”) module 808 communicatively connected via acommunication infrastructure 810. While an exemplary computing device800 is shown in FIG. 8, the components illustrated in FIG. 8 are notintended to be limiting. Additional or alternative components may beused in other embodiments. Components of computing device 800 shown inFIG. 8 will now be described in additional detail.

Communication interface 802 may be configured to communicate with one ormore computing devices. Examples of communication interface 802 include,without limitation, a wired network interface (such as a networkinterface card), a wireless network interface (such as a wirelessnetwork interface card), a modem, an audio/video connection, and anyother suitable interface.

Processor 804 generally represents any type or form of processing unitcapable of processing data or interpreting, executing, and/or directingexecution of one or more of the instructions, processes, and/oroperations described herein. Processor 804 may direct execution ofoperations in accordance with one or more applications 812 or othercomputer-executable instructions such as may be stored in storage device806 or another computer-readable medium.

Storage device 806 may include one or more data storage media, devices,or configurations and may employ any type, form, and combination of datastorage media and/or device. For example, storage device 806 mayinclude, but is not limited to, a hard drive, network drive, flashdrive, magnetic disc, optical disc, RAM, dynamic RAM, other non-volatileand/or volatile data storage units, or a combination or sub-combinationthereof. Electronic data, including data described herein, may betemporarily and/or permanently stored in storage device 806. Forexample, data representative of one or more executable applications 812configured to direct processor 804 to perform any of the operationsdescribed herein may be stored within storage device 806. In someexamples, data may be arranged in one or more databases residing withinstorage device 806.

I/O module 808 may include one or more I/O modules configured to receiveuser input and provide user output. One or more I/O modules may be usedto receive input for a single virtual reality experience. I/O module 808may include any hardware, firmware, software, or combination thereofsupportive of input and output capabilities. For example, I/O module 808may include hardware and/or software for capturing user input,including, but not limited to, a keyboard or keypad, a touchscreencomponent (e.g., touchscreen display), a receiver (e.g., an RF orinfrared receiver), motion sensors, and/or one or more input buttons.

I/O module 808 may include one or more devices for presenting output toa user, including, but not limited to, a graphics engine, a display(e.g., a display screen), one or more output drivers (e.g., displaydrivers), one or more audio speakers, and one or more audio drivers. Incertain embodiments, I/O module 808 is configured to provide graphicaldata to a display for presentation to a user. The graphical data may berepresentative of one or more graphical user interfaces and/or any othergraphical content as may serve a particular implementation.

In some examples, any of the facilities described herein may beimplemented by or within one or more components of computing device 800.For example, one or more applications 812 residing within storage device806 may be configured to direct processor 804 to perform one or moreprocesses or functions associated facilities 302 through 306 of system300. Likewise, storage facility 308 of system 300 may be implemented bystorage device 806 or a component thereof.

In the preceding description, various exemplary embodiments have beendescribed with reference to the accompanying drawings. It will, however,be evident that various modifications and changes may be made thereto,and additional embodiments may be implemented, without departing fromthe scope of the invention as set forth in the claims that follow. Forexample, certain features of one embodiment described herein may becombined with or substituted for features of another embodimentdescribed herein. The description and drawings are accordingly to beregarded in an illustrative rather than a restrictive sense.

1. A system comprising: a processor; and a memory communicativelycoupled to the processor and storing instructions executable by theprocessor to: determine a distance, along a line extending from aviewpoint of an endoscope in a three-dimensional space, between a pointon an anatomical surface visible to the endoscope and a point on anembedded anatomical object visually occluded from the viewpoint of theendoscope by the anatomical surface; determine, based on the determineddistance, a display parameter for a pixel of an image, the pixelcorresponding to the point on the anatomical surface; assign thedetermined display parameter to the pixel of the image; provide theimage for display, the image comprising a view of the anatomical surfacefrom the viewpoint of the endoscope and a visualization of the embeddedanatomical object, the visualization indicating a depth of the embeddedanatomical object from the anatomical surface; and provide a usercontrol to adjust a prominence of the visualization of the embeddedanatomical object in the image.
 2. The system of claim 1, wherein theinstructions are executable by the processor to determine the displayparameter using a defined function that specifies how the displayparameter changes as the determined distance changes.
 3. The system ofclaim 1, wherein: the display parameter comprises a color for the pixelof the image; and the instructions are executable by the processor todetermine, based on the determined distance, the color for the pixel ofthe image using a color blending function in which a color associatedwith the point on the embedded anatomical object is blended with a colorassociated with the point on the anatomical surface to determine thecolor for the pixel of the image.
 4. The system of claim 3, wherein thecolor associated with the embedded anatomical object is given moreweight in the color blending function when the determined distance isrelatively shorter and less weight in the color blending function whenthe determined distance is relatively longer.
 5. The system of claim 4,wherein the color blending function specifies how the weight given tothe color associated with the embedded anatomical object changes fordifferent values of the determined distance.
 6. The system of claim 1,wherein the instructions are executable by the processor to: determine,based on the determined distance, a blend parameter for the point on theembedded anatomical object; and determine, based on the blend parameterfor the point on the embedded anatomical object, the display parameterfor the pixel of the image.
 7. The system of claim 6, wherein theinstructions are executable by the processor to determine the blendparameter for the point on the embedded anatomical object using afunction that specifies how the blend parameter changes for differentvalues of the determined distance.
 8. The system of claim 1, wherein theinstructions are executable by the processor to determine the displayparameter based on a maximum distance beyond which the point on theembedded anatomical object is not visually represented by the pixel ofthe image and within which the point on the embedded anatomical objectis visually represented by the pixel of the image.
 9. The system ofclaim 1, wherein the instructions are executable by the processor todetermine the display parameter based on a minimum visibility parameterallowed for the point on the embedded anatomical object.
 10. The systemof claim 1, wherein the user control is configured to facilitate userinput to adjust, in real time during a surgical procedure, a settingused to determine the display parameter based on the determineddistance.
 11. The system of claim 1, wherein the instructions areexecutable by the processor to provide an additional user controlconfigured to facilitate user input to toggle, in real time during asurgical procedure, between display modes that include: a mode forvisualization of only surface anatomy visible to the endoscope; a modefor concurrent visualization of surface anatomy visible to the endoscopeand embedded anatomy; and a mode for visualization of only embeddedanatomy.
 12. A computer-assisted surgical system comprising: at leastone physical computing device communicatively coupled to a stereoscopicendoscope and a display device, the at least one physical computingdevice configured to: determine a distance between a point on ananatomical surface visible to the endoscope and a point on an embeddedanatomical object visually occluded from a viewpoint of the endoscope bythe anatomical surface; determine, based on the determined distance, adisplay parameter for a pixel of an image, the pixel corresponding tothe point on the anatomical surface; provide the image for display bythe display device, the image including the pixel displayed inaccordance with the display parameter, the image comprising a view ofthe anatomical surface from the viewpoint of the endoscope and avisualization of the embedded anatomical object; and provide a usercontrol to adjust a prominence of the visualization of the embeddedanatomical object in the image.
 13. A method comprising: determining, byan anatomical structure visualization system, a distance, along a lineextending from a viewpoint of an endoscope in a three-dimensional space,between a point on an anatomical surface visible to the endoscope and apoint on a modeled anatomical object visually occluded from theviewpoint of the endoscope by the anatomical surface; determining, bythe anatomical structure visualization system and based on thedetermined distance, a display parameter for a pixel of an image, thepixel corresponding to the point on the anatomical surface; assigning,by the anatomical structure visualization system, the determined displayparameter to the pixel of the image; providing the image for display,the image comprising a view of the anatomical surface from the viewpointof the endoscope and a visualization of the embedded anatomical object,the visualization indicating a depth of the embedded anatomical objectfrom the anatomical surface; and providing a user control to adjust aprominence of the visualization of the embedded anatomical object in theimage.
 14. The method of claim 13, wherein the determining of thedisplay parameter comprises using a defined function that specifies howthe display parameter changes as the determined distance changes. 15.The method of claim 13, wherein: the display parameter comprises a colorfor the pixel of the image; and the determining of the display parametercomprises determining, based on the determined distance, the color forthe pixel of the image using a color blending function in which a colorassociated with the point on the embedded anatomical object is blendedwith a color associated with the point on the anatomical surface todetermine the color for the pixel of the image.
 16. The method of claim13, wherein the determining of the display parameter comprises:determining, based on the determined distance, a blend parameter for thepoint on the embedded anatomical object; and determining, based on theblend parameter for the point on the embedded anatomical object, thedisplay parameter for the pixel of the image.
 17. The method of claim13, wherein the determining of the display parameter is further based ona maximum distance beyond which the point on the embedded anatomicalobject is not visually represented by the pixel of the image and withinwhich the point on the embedded anatomical object is visuallyrepresented by the pixel of the image.
 18. The method of claim 13,wherein the determining of the display parameter is further based on aminimum visibility parameter allowed for the point on the embeddedanatomical object.
 19. The method of claim 13, wherein then user controlis configured to facilitate user input to adjust, in real time during asurgical procedure, a setting used to determine the display parameterbased on the determined distance.
 20. The method of claim 13, furthercomprising providing an additional user control configured to facilitateuser input to toggle, in real time during a surgical procedure, betweendisplay modes that include: a mode for visualization of only surfaceanatomy visible to the endoscope; a mode for concurrent visualization ofsurface anatomy visible to the endoscope and embedded anatomy; and amode for visualization of only embedded anatomy.